Skip to content

Phase 1: Inventory, the classification rule, and the docs-link drift guard - #1381

Merged
neuromechanist merged 4 commits into
feature/issue-1336-epic-agent-docsfrom
feature/issue-1339-phase1-inventory-classification
Sep 13, 2026
Merged

neuromechanist merged 4 commits into
feature/issue-1336-epic-agent-docsfrom
feature/issue-1339-phase1-inventory-classification

Conversation

@neuromechanist

Copy link
Copy Markdown
Contributor

Summary

Groundwork for ADR 0057, which reset this epic's rules after phase 0 showed the gate controls who is served rather than who can read. Phase 1 decides what moves, writes the rule down so a second person can apply it, and builds the guard that keeps later phases' docs pointers honest. It moves nothing.

Closes #1339. Part of epic #1336.

Three deliverables

ADR 0058, the classification rule. Move an item if it names a code-execution or credential path concretely (host + path + ref + cadence), names where a credential or a full data copy lives, describes an incomplete security migration, or is an assembled operational map rather than a single fact. Assembly is the trigger.

Half the ADR is what does not move, because over-collection is the likelier failure. ADRs, architecture, contracts and the API surface stay. Every blast-radius warning stays, explicitly, because those prevent the likelier harm -- a contributor breaking something by accident -- so tidying them behind the gate later makes the system less safe. Material whose only protection would be obscurity against someone already holding an admin key stays too; the control there is key hygiene, not classification.

A destination column on .context/README.md, extending the existing index rather than adding a parallel one. Destination only, no per-file rationale: a list of destinations is not a target list. Result is 4 private, 32 public, matching the small move the harm evaluation on #1339 predicted.

The sweep also found four documents that were in .context/ but in no list at all, so the index could not classify what it could not see. The Historical bullet list became a table for the same reason: historical does not mean harmless, and one completed plan enumerates the staging worker's entire secret inventory by name.

test/docs-links.unit.test.ts, the pointer guard. Two non-obvious things, both measured:

  • Resolution is a slug map, not path arithmetic. slug: frontmatter decouples source path from published URL in both directions. platform/zarr.md declares slug: platform/zarr/mental-model while /platform/zarr/ is served by platform/zarr/index.md, so candidate-path guessing falsely passes one and falsely fails the other.
  • It reads a git ref, never the working tree. Two exploration passes disagreed about whether any page uses slug: at all, because one read a sibling checkout parked on a feature branch missing twenty-odd pages. A checker whose answer depends on which branch someone left checked out reports drift that is not there, and the blame lands on the checker.

The CI wiring is half the change

No paths filter matched AGENTS.md or .rules/**, so a PR editing only those ran zero jobs and this test would never have run on exactly the change it exists to police. That is the third time for this gap; the shared/** and decisions/** versions both shipped green and vacuous before being found, and test.yml records both. Adds an agentdocs filter, gates lint and unit-pure on it, and sparse-checks out the docs content tree so CI compares against something rather than skipping.

Test plan

  • 11 tests in the new guard, mutation-checked three ways: a broken pointer goes red and names the offender; a declared checkout that is missing fails rather than skipping; a directory that is not a docs checkout fails too.
  • Confirmed the file classifies into unit-pure (required, offline, deploy-gating) and not integration-dev, by running the workflow's own content grep against it.
  • test/workflow-yaml.test.ts green, which is what asserts declared filters and job outputs agree.
  • test/adr-index.unit.test.ts green.
  • Verified every .context document is classified: 36 of 36, none missing, none naming a file that is not on disk.
  • bun run lint and tsc --noEmit clean (pre-commit runs both).

Not in this PR

Moving any file, which is phase 3 along with the private repo, the credential exchange, and correcting robots.txt and docs ADR 0002 -- both true today and false only once the source moves. Rewriting AGENTS.md is phase 5, after phase 2 makes the URLs retrievable.

ADR 0057 makes the docs site the canonical retrieval surface, so AGENTS.md and
.rules/ stop describing things and start pointing at them. A pointer that
resolves to nothing is then the reference material being unreachable for the
agent told to read it. starlight-links-validator covers the docs repo's own
internal links; nothing covered this direction, and it would not catch it
anyway, since it ignores absolute URLs by default.

Two things here are not obvious and both were measured.

Resolution is a slug map, not path arithmetic. `slug:` frontmatter decouples
source path from published URL in both directions: platform/zarr.md declares
slug platform/zarr/mental-model while /platform/zarr/ is served by
platform/zarr/index.md, so guessing candidate paths falsely passes one and
falsely fails the other.

The comparison reads a git ref, never the working tree. Two exploration passes
disagreed about whether any page uses `slug:` at all, because one read a sibling
parked on a feature branch missing twenty-odd pages. A checker whose answer
depends on which branch someone left checked out reports drift that is not
there.

CI wiring is half the change. No paths filter matched AGENTS.md or .rules/**, so
a PR editing only those ran zero jobs and this test would never have run on
exactly the change it polices. That is the third time for this gap; the shared/**
and decisions/** versions both shipped green and vacuous before being found. Adds
an agentdocs filter, gates lint and unit-pure on it, and sparse-checks out the
docs content tree so CI compares against something rather than skipping.

Tested: 11 tests. Mutation-checked three ways -- a broken pointer goes red and
names the offender, a declared checkout that is missing fails rather than
skipping, and a directory that is not a docs checkout fails too. Confirmed the
file classifies into unit-pure, not integration-dev.
ADR 0057 decided admin-only material goes private at source but did not say what
counts as admin-only, and that judgment is about to be applied to roughly 9,900
lines across four public repos by more than one person. An unwritten rule gives
a different answer every time.

The rule: move an item if it names a code-execution or credential path concretely
(host plus path plus ref plus cadence), names where a credential or a full data
copy lives, describes an incomplete security migration, or is an assembled
operational map rather than a single fact. Assembly is the trigger, and it is the
part people get wrong.

Half the ADR is what does NOT move, because over-collection is the likelier
failure. ADRs, architecture, contracts and the API surface stay. Every
blast-radius warning stays, and the ADR says so explicitly: those prevent the
likelier harm, which is a contributor breaking something by accident, so tidying
them behind the gate later makes the system less safe rather than more. Material
whose only protection would be obscurity against someone already holding an admin
key stays too; the control there is key hygiene, not classification.

Calibrated per the owner: no credential value is exposed, this is not an
incident, public documentation stays public because that is what the project is
for, and the expected result is a small move.
The inventory half of phase 1. Extends the existing index rather than adding a
parallel document, since a second index is the duplication this epic exists to
remove.

One Destination column carrying the ADR 0058 classification: public, private, or
stays-with-code. Destination only, with no per-file rationale -- a list of
destinations is not a target list, and the reasoning belongs in the ADR as a
general rule. Nothing moves here; phase 3 does that.

Four documents were in .context/ but in no list at all, so the index could not
classify what it could not see. They are now under a Design notes heading. The
Historical bullet list became a table for the same reason: historical does not
mean harmless, since a completed plan can still name things that are current,
which is exactly the case for one of them.

Result is 4 private and 32 public, matching the small move the harm evaluation
predicted. Verified that every .context document is classified, with none
missing and none naming a file that is not on disk.
@github-actions

github-actions Bot commented Sep 13, 2026 •

Copy link
Copy Markdown

📦 Package Preview

Your package is ready to test!

bun install -g nemar-cli@PR1381

Version: 0.10.4-PR1381.568b06f
Commit: 568b06f


🤖 Updated on each push.

The most serious one was the same defect this PR exists to fix, one level up.
`paths-ignore: '*.md'` on both triggers meant every ROOT markdown file skipped
the Tests workflow entirely, before the paths filter was ever evaluated -- so
the new agentdocs filter could never fire for AGENTS.md, CLAUDE.md or README.md,
the files it names. Confirmed on dev commit 02b4218, a CHANGELOG-only change
with no Tests run at all. Narrowed to CHANGELOG.md, which is genuinely inert.

README.md turned out to carry four of the repo's five page pointers and was in
neither the filter nor the trigger. Rather than patch the allowlist again, the
scan now asks git which tracked files mention the host. That covers README, the
.context tree this epic is adding pointers to, and the two URLs quoted to users
in refusals from backend and CLI source, none of which an allowlist had.

Anchors and query strings were false failures: /cli/commands/#nemar-auth-login
names a page that exists and was looked up whole. Deep links are the natural
pointer form once AGENTS.md points rather than describes, so the first person to
write one would have learned the guard was wrong.

Three ways the guard could pass while checking nothing, all now closed. git()
collapsed "no match" into "could not run", so a broken grep read as a repo with
no slug overrides and the test meant to catch that read the same empty result
and passed. The override test re-ran its own search with a looser pattern, so an
extraction bug cancelled out on both sides, and its early return went green
rather than skipping when overrides vanished -- measured against a fixture, not
theorized. The pointer side had no floor, so a scan returning no files passed.

Also: slug values are unquoted before use, since quoted YAML is legal and filed
the real page under a key wearing quotation marks; slug detection is scoped to
the frontmatter block, since a fenced YAML example on a docs site that documents
frontmatter would otherwise be read as an override; origin/main is preferred
over a developer's stale local main, which was four commits and three pages
behind; and the test no longer exports, which tripped biome.

ADR 0058 said "9,900 lines across four public repositories". That figure was for
this repo alone and was itself wrong: 19,385 committed lines here, about 23,100
across the four. Its list of what moves double-counted one file and named a
document that does not exist; it now names the four files one to one. Two more
.context entries had no destination, a script and a file in plans/, both missed
by a markdown-only top-level sweep.

Tested: 16 tests. Mutation-checked again -- a broken pointer goes red naming the
file, an anchored pointer to a real page stays green, and CI with a declared
checkout that has no resolvable main fails rather than skipping.
@neuromechanist
neuromechanist merged commit fb58aae into feature/issue-1336-epic-agent-docs Sep 13, 2026
12 checks passed
@neuromechanist
neuromechanist deleted the feature/issue-1339-phase1-inventory-classification branch September 13, 2026 12:39

This branch was previously deployed

1 inactive deployment
npm — 568b06f1 Deployed Sep 13, 2026 by neuromechanist via publish #2524
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant